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Abstract 



In this paper, we study the problem of channel simulation via interactive communication, known as the coordina- 
tion capacity, in a two-terminal network. We assume that two terminals observe i.i.d. copies of two random variables 
and would like to generate i.i.d. copies of two other random variables jointly distributed with the observed random 
variables. The terminals are provided with two-way communication links, and shared common randomness, all at 
limited rates. Two special cases of this problem are the interactive function computation studied by Ma and Ishwar, 
and the tradeoff curve between one-way communication and shared randomness studied by Cuff. The latter work had 
inspired Gohari and Anantharam to study the general problem of channel simulation via interactive communication 
stated above. However only inner and outer bounds for the special case of no shared randomness were obtained in 

fvj ' their work. In this paper we settle this problem by providing an exact computable characterization of the multi-round 

problem. To show this we employ the technique of "output statistics of random binning" that has been recently 

CO . developed by the authors. 

CN 



I. Introduction 

The minimum amount of interaction needed to create dependent random variables is an operational way to 
quantify the correlation among random variables. Wyner considered the problem of remote reconstruction of two 
dependent random variables by two terminals which are provided with shared randomness at a limited rate |fl]. 
He used this approach to measure the intrinsic common randomness between two random variables. An alternative 
characterization of Wyner's common information as an extreme point of a channel simulation problem was provided 
in iff) ■ In this setup, a terminal who observes i.i.d. copies of X sends a message at rate R\ to a remote random 
number generator (decoder) that produces i.i.d. copies of another random variable Y that is jointly distributed with 
X. The total variation distance between the achieved joint distribution and the i.i.d. distribution induced by passing 
X through a DMC channel p(y\x) should be negligible. In other words, the generated distribution and the i.i.d. 
distribution should be statistically indistinguishable. Shared common randomness exists between the two parties at 
a limited rate Rq. Cuff found the tradeoff between Rq and i?i showing that when Rq — the minimum admissible 
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Fig. 1. Channel simulator model: collective forward and backward rates satisfy nR\2 > ^li-odd,H(Ci) and nR.21 > ^i. even H(Ci) 
respectively. 

rate for R\ is the Wyner's common information; and when Rq = 00, the minimum admissible rate for i?i is the 
mutual information between X and Y (this special case was already shown in J5]). 

This setup was generalized in (4) by assuming that two terminals have access to i.i.d. copies of X\ and X2 
respectively and would like to generate i.i.d. copies of Y\ and Y2. Instead of a one-way communication, now the 
terminals are provided with a two-way communication at rates R12 and R21 (see Fig. 1). They can use up these 
two resources in r rounds of interactive communications as they wish (i.e. we only impose the constraint that 
J2i dd.H(Ci) is less than or equal to nR\2 where H(Cj) is the entropy of the message sent from terminal 1 to 
terminal 2 at round i; a similar statement holds for i?2i)- Inner and outer bounds on R12 and R21 were derived in 
the special case of no shared common randomness |EQ. In this paper we completely solve this problem under both 
the strong and empirical coordination models. Strong coordination demands a total variation converging to zero. On 
the other hand, empirical coordination only demands closeness of the empirical distribution of the generated random 
variables and the i.i.d. ones [5| (See Section UT1 for a detailed description of these two models). Our result relates 
to the literature of coordinating distributed controllers to carry out some joint action (see e.g. J5], 0~2]) since the 
generated random variables can be thought of as coordinated actions. Also, our result has implications in quantum 
information theory. Finding the communication cost of simulating non-local correlations has been subject to many 
studies where the goal is to simulate an arbitrary bipartite box p(yi,y2\xi,X2). Our result in this paper implies an 
asymptotic information theoretic characterization of the communication cost (that serves as a lower bound to the 
one-shot communication complexity formulation whose characterization remains an open problem; see [6|). As a 
future work along these lines, it would be interesting to find the entanglement assisted version of our results. 

This paper is organized as follows: in the next subsection we describe the main proof technique at an intuitive 
level. In Section [TT] we define the problem and in Section [III] we state the main results followed by proofs in Sections 
|TV]and|V] 

Notation: In this paper, we use Xg to denote (Xj : j G S). we use p v A to denote the uniform distribution over 
the set A and p(x n ) to denote the i.i.d. pmf IliLiP^j)' un l ess otherwise stated. The total variation between two 
pmf's p and q on the same alphabet X , is denoted by |[p(x) — ^(x)!^. 
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Remark 1 Similar to /|2]? in this work we frequently use the concept of random pmfs, which we denote by capital 
letters (e.g. Px)- For any countable set X let A x be the probability simplex for distributions on X. A random pmf 
Px is a probability distribution over A x . In other words, if we use il to denote the sample space, the mapping 
w6!)h) P x (x; lo) is a random variable for all x 6 X such that Px {x; to) > and ^2 X Px {x; w) = 1 for all lo. 
Thus, lo <— > Px(;',lj) is a vector of random variables, which we denote by Px- We can definite Px.y on product 
set X x y in a similar way. We note that we can continue to use the law of total probability with random pmfs 
(e.g. to write Px{x) = ^2 Pxy(x, y) meaning that Px{x;lo) = £^ PxY{x,y,Lo) for all lo) and the conditional 
probability pmfs (e.g. to write Py\x{v\ x ) = X p ( x ) meaning that PY\x{y\ x \Lo) = X p ( X V J? for all lo). 

A. Description of the proof technique 

Our proof is based on the technique of "output statistics of random binning" (OSRB) that has been recently 
developed in ifTTI . To explain the technique we begin by describing the resolvability lemma used by Cuff J2] 
Lemma 6.1], and originally proved by Wyner. We report this lemma in a slightly different form that suits our 
purpose. Although we do not use this lemma in this work, since it is very central to the achievability proof of [2], 
we illustrate how this lemma can be proved using the OSRB approach. 

To discuss the resolvability lemma of [2 Lemma 6.1], let us fix some p(x,y). Roughly speaking the lemma 
states that one can find 2 nR sequences in X n , namely x n (l), X n (2), ■ ■ ■ , x n (2 nR ), such that if we choose one of 
these sequences at random and pass it through the DMC channel p(y\x) we get an output sequence that is almost 
i.i.d. according to p(y), as long as R > I(X: Y). We can restate this lemma by letting M to be a random variable 
whose alphabet is M. = [1 : 2 n/ ( X;y )], and assuming that X n (M) is transmitted over the DMC channel q(y\x). 
To prove this lemma in the traditional way one would construct a random codebook parametrized by a random 
variable B. Every choice of B = b corresponds to a particular codebook (particular set of sequences in the X n 
space). The probability distribution imposed on the y n space depends on the value of B, which is itself random. 
Therefore we use the capital letter Py™ to denote the random p.m.f. induced on y n , by the random codebook. To 
show the above lemma one would need to show that the expected value of the total variation distance between the 
probability measure y n \-> P(y n ) and the i.i.d. distribution is small. Therefore there exists B — b where the total 
variation distance is small. Indeed this is the way that Cuff proves this lemma in [8 Lemma 19]. 

To illustrate the proof of this lemma using the OSRB approach, one would need to start from n i.i.d. copies of 
X n and Y n from the given p(x, y). Random variables B and M are identified as random binnings of X n at rates 
2 nR and 2 nR respectively. Note the conceptual change is in starting from the i.i.d. distribution and then defining B 
as a function of X n . It is proved that if R < H(X\Y), B is almost independent of Y n , Therefore, for almost any 
choice of B = b, the distribution of Y n conditioned on B = b is almost i.i.d. On the other hand, if R + R > H(X), 
X n will be a function of (M, B) with high probability by the Slepian-Wolf. We are interpreting B and M as 
two messages coming from two encoders both observing X n . These imply that one can find B = b such that the 
conditional law y n n- p(y n \B — b) is close to the i.i.d. distribution, and at the same time X n is almost a function 
of M conditioned on B = b. All the approximations in this intuitive argument can be made accurate. 
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The crucial departure from the traditional argument was our treatment of random variable B. As discussed in 
ifTTI . the randomness in generating a random codebook is generally conceived of a common randomness shared 
among the terminals in a problem. However, we are changing the order by first generating i.i.d. distributions and 
then treating B as a random binning on this product i.i.d. space. 

II. Problem Statement 

Two terminals observe i.i.d. copies of sources X\,X% (taking values in finite sets X\ and X2 and having a 
joint pmf q(xi, X2)) respectively. A random variable 00 which is independent of X5, 2 i = X^X^ and is uniformly 
distributed over [1 : 2 nRo ] represents the common randomness provided to the terminals. Given an arbitrary r G N, 
an (n, Ro, R12, R21) channel simulation code for simulating a channel with r interactive rounds of communications, 
consists of 

> a set of r randomized encodings specified with the conditional pmf's p enci (ci|c[ 1 . i _ 1 ]x"a;) for odd numbers 
i G [1 : r] and p enC2 (ci\c[i :i _i]X2 u) for even numbers i G [1 : r], where Cj denotes the communication of the 
i-th round, 

• two randomized decoders p deci (y"|c[i :T .]x"a;) and p dec - 2 (y^c^.^x^oj), 
such that 



- V H{d) <R 12 , -V H(d) < R21. 
n *-^ n *-~* 

i:odd i:even 



Definition 1 Given a channel with transition probability q(y [i : 2\\x[i : 2\), a rate tuple (Ro, R12, R21) is said to 
be achievable if there exists a sequence of (n, Rq, R12, R21) channel simulation codes, such that the total vari- 
ation between the probability P(yl[.2]y x ]\-2]) induced by the code and the i.i.d. repetitions of the desired pmf 
q(y[i:2]\x[i:2])q(x[i-2]) vanishes as n goes to infinity, that is 



lim 

n— ►oo 



0. (1) 



P(y[l:2] X [l:2]) ~ Yl ^(^[1:2] ,i X [l:2],i) 

4=1 

Definition 2 The simulation rate region is the closure of all the achievable rate tuples (Rq, R12, i?2i)- 

Remark 2 In the special case r = 1, [Vi = X2 = 0, our problem reduces to the one considered by Cuff in /Hty. 

Remark 3 Observe that if Y\ = /i(^[i : 2]) ond I2 = /2(-^[i:2l) are deterministic functions, the total variation 
constraint of eq. (Q3 reduces to 

\\mj (17 = h(X^.2)l Y? = /2(*[i :2] )) = I- 

Thus our problem reduces to the problem of interactive function computation considered in ftTj. 

Definition 3 (Empirical coordination |5|) Assume that instead of simulating the channel q(yu;2]\%[i:2])> me de- 
mand is to find encoders and decoders such that the output sequences ifjoi ore jointly typical with the inputs 
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Xr™. 2 |, with high probability. In this case, condition (Q3 should be replaced by the following condition: 



lim p 



PX ri:2] y [? : 2] " ^[1:2] ^[1:2] , 



>e =0, (2) 



where px n Y n is the empirical distribution of the pair (XS. 2 i, Yr™, 2 -,) induced by the chosen code. 

Remark 4 It can be shown that if a sequence of codes satisfies the channel simulation condition (Q3, then it 
also satisfies the empirical coordination constraint (|2]). On the other hand it was shown in [5, Theorem 2] that the 
empirical rate region does not depend on the amount of common randomness, that is, if (Ro, R12, R21) is achievable 
for empirical coordination, then (0, R12, R21) is also achievable. These two facts imply that the achievability of a 
pair of (-R12, -R21) for empirical coordination can be proved indirectly through the achievability proof for channel 
simulation in the presence of an unlimited common randomness. In /|5y, it was conjectured that this relation is two- 
sided, i.e. the rate regions for empirical coordination and channel simulation with unlimited common randomness 
are equal. 

III. Main Results 
Theorem 1 (Channel Simulation) The simulation rate region is the set S{r) of all non-negative rate tuples 
(Ro,Ri 2 ,R 2 i), far which there exists p(fi, • • • , /r)*[i:2])J/[i:2l) G ^( r ) such that 

R12 > I{Xi;F[ 1:r ]\X2), 
R21 >I{X 2 ;F [1:r] \X 1 ), 
Ra + Ri2 > I(X l ;F [1:r] \X 2 ) + I(F i; Y [1:2] \X [1:2] ), 
Ro + R12 + R21 > I{Xi;F[ 1:r ]\X 2 ) + I(X 2 ;F[ 1:r ]\Xi) 

+ I(F [1:r] ;Y [1:2] \X [1 .. 2] ), (3) 

where T{r) is the set ofp(fi, ■■■ , f r , X[ 1:2 ] , y[i :2 ]) satisfying 

X[i:2],Y [1:2] ~ g(£[l:2])g(2/[l:2]|Z[l:2]), 

Fi—Fti : i_i\Xi — X 2 , if i is odd, 
F i —F[ 1 . i _-r i X 2 - Xi, if i is even, 

Y\ — F[i :r ]Ai — X 2 Y 2 , 

Y 2 -F [1:r] X 2 -X l Y 1 
j-i 

V^ : \F t \ < 1^1 l^ll^iH^ I J] 1^1 + L W 

Corollary 1 (Interactive function computation |7|) Assume that the desired channel is deterministic, that is, 
Y\ = fi(Xri :2 i) and Y 2 = /2(-^[i:2])- Setting Rq = in Theorem [7] gives the following full characterization 
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of the rate region of reliable interactive computation, 

K(r) = {3F [1:r] :R 12 > I(X 1 ;F [1 .. r] \X 2 ) 

R21 >I(X 2 ;F [1:r] \X 1 ) 

Fi — F[i : ,-_i]Xi — X 2l if i is odd, 

Fi - F [1 . l _ 1] X 2 - X x , if i is even, 

H(Y 2 \F [Ur] X 2 ) = HiY.lF^X,) = 0}. 
Theorem 2 (Empirical coordination) The empirical coordination rate region is the set of all non-negative rate 
pairs (-R12, R21), for which there exists p(fi, • • • , f r ,%[i:2]> J/[i:2l) G T(r) (defined in Theorem^ such that 

-Rl2 > /(XijF^I-X'g), 

R2i>I(X 2 ;F [1:r] \X 1 ). (5) 

IV. Proof of the achievability 

A. Review of the output statistics of random binning 

Let (X[ l:T ],Y) be a DMCS distributed according to a joint pmf px ri . x] ,y on finite sets. A distributed random 

binning consists of a set of random mappings Bi : XJ 1 — > [1 : 2 nRi ], i E [1 : T], in which Bi maps each sequence of 

A?" uniformly and independently to [1 : 2 nRi ]. We denote the random variable Bt{X^) by B t . A random distributed 

binning induces the following random pmf on the set #r™. T i X J 7 " X IIt=i[l : 2 niit ], 

r 
P(x[ l 1:T]l y",6 [1:T] )=p(a ; f 1:T] ,^)ni{S i (x t ™) = 6 t }. 

Theorem 3 ( Bill ) If for each S C [1 : T], f/ie following constraint holds 

Y,Rt<H(X s \Y) 1 

then as n goes to infinity, we have 



(6) 



ies 



E 



i 5 b",i|l:n)-#)IIP|l:2-.](y 



0. 



(7) 



We now consider another region for which we can approximate a specified pmf. This region is the Slepian-Wolf 
region for reconstructing -XTJ.^i in the presence of (B 1: t, Y n ) at the decoder. As in the achievability proof of the 
iflOl Theorem 15.4.1], we can define a decoder with respect to any fixed distributed binning. We denote the decoder 
by the random conditional pmf P sw (xV 1 ^^ \y n , &[i : t]) (note that since the decoder is a function, this pmf takes 
only two values, and 1). Now we write the Slepian-Wolf theorem in the following equivalent form. See [11 1 for 
details. 
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Lemma 1 ([XI]) If for each S C [1 : T], the following constraint holds 

J2Rt>H(X s \X S o,Y), (8) 



tes 
then as n goes to infinity, we have 



E 



P\ x \X;T\iV > X [1:T\) ~ P\ x [XxT\iV )^-\ x [l:T] ~ X [1:T]S 



0. 



Definition 4 For any random pmfs Px and Qx on X, we say Px ~ Qx if E ||Px — Qx Hi < & Similarly we use 
Px w Qx for two (non- random) pmfs to denote the total variation constraint \\px — Qx\\i < e - 

Lemma 2 ([11]) We /zave 

1) HpxPyix-ffxPyixlli = lbx-?x|li 

IIpx — ffxlli < \\pxPy\x - qxqy\x\\ 1 

2) If pxPy\x ~ 9x9y|x> f ^ en f/zere exists x G A' ^mc/; f/iaf py\x=x ~ <?y|x=x- 

3) //Px » Qx anrf PxPyix » PxQy\x, then P x P Y \x »' QxQy|x- 

B. Achievability proof of Theorem Q] 

We use a combination of the Slepian-Wolf theorem and Theorem [3] to prove Theorem Q] In this section we 
present a simplified version of the proof. A rigorous one is provided in the appendix. 

The proof is divided into three parts. In the first part we introduce two protocols each of which induces a pmf 
on a certain set of r.v.'s. The first protocol has the desired i.i.d. property on (Xr". 2 ,, Vr" 2 i) but leads to no concrete 
coding algorithm. However the second protocol is suitable for construction of a code, with one exception: the second 
protocol is assisted with an extra common randomness that does not really exist in the model. In the second part 
we find constraints on Ro,Ri2,R2i implying that these two induced distributions are almost identical. In the third 
part of the proof, we eliminate the extra common randomness given to the second protocol without disturbing the 
pmf induced on the desired random variables (X,™. 2 , , Y,™ . 2 ,) significantly. This makes the second protocol useful 
for code construction. 

Part (I) of the proof: Take an arbitrary p{f[\-. r ]i x [i-.2]i J/[i:2l) G T{ r )- We define two protocols each of which 
induces a joint distribution on random variables that are defined during the protocol. 

Protocol A. We begin by describing a random binning strategy that we will use when defining Protocol A. 

Random Binning: Let {Fil. r ],XJ[. 2 i 7 Yft. 2 A be i.i.d. and distributed according to p(f[i :r ], X[i:2]>2/[i:2])- Since 
P(/[i:r],£[i:2],2/[i:2]) € T(r), it factors as 



P(afl:2]) 



IlK/fl/[L-i]4) 2 ) 



P(yri/[":r]^)p(y 2 n |/[l:r]*2)- 



Consider the following random binning: 

• To each sequence /", assign uniformly and independently three bin indices b\ G [1 : 2 nRl ], fei G [1 : 2 nRl ] 
and a; G [1 : 2"^°]. 
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• For i G [2 : r], to each sequence (/", • • • , /"), assign uniformly and independently two bin indices bi € [1 : 
2 nRi ] and k t E [1 : 2 nfl «]. 
Furthermore, for i S [1 : r], we consider the Slepian-Wolf decoder for recovering /" from (/r". i _ 1 i ,bi,ki,ui, xl l i+1) ) 
and denote it by P sw (f™\bi, ki, w, /r"^ _ 1 i,X/ l i+1 - ) ). Note that we denote the estimates of /" by /". The rate 
constraints for the success of these decoders will be imposed later, although these decoders can be conceived even 
when there is no guarantee of success. 

We define /™ T for terminal 1, i.e. T\, and /™ T for terminal 2, i.e. T 2 as follows: 

/" for odd i 
/" for even i, 



rn 
Jijl 

and 



rn 

■li,T 2 — 



/" for even i 
/" for odd i. 
The random pmf induced by the random binning, denoted by P, can be expressed as follows: 



-P( x [l:2] >/[" r] ' ^[l:r] > fylrr] 7 W, J/^ :2 ] , /[l^l.Tj ' /[™:r],T 2 ) 



= P(^fl:2]) 



= P(®fi:2l) 



rip(/"i/[i-i]'4) 2 ) p ( 6 - fc -^i/"'/[":-i]) 



pSWr (/"Tc <+1 , 2 l 6 *'*i. W *'/[l:i-l]' a; ?i+l) a ) 1 {^T (i)a = /"} Ml/ll/il^^W^I/^,^) 



nP^i.Wil/^j.^JP^.fcilfti.Wi,/^.!],^ 



.i=l 



(9) 
where (1)2 := i mod 2 and wi = w, and w» is a constant variable for i > 2. 

Protocol B. Given some p(/ri :r ], £[i:2l) J/[i:2l) <= ^"( r )> we define Protocol B as follows: In this protocol we 
assume that the terminals have access to the shared randomness B[i :r i where Bri :r ] are mutually independent r.v.'s 
and uniformly distributed on 111=1 [1 : 2 nRt ). R.v. 10 is also used for the common randomness (it is independent 
of B[ 1:r j). The shared randomness Bri :r i does not really exist in the real model, and we will get rid of it later. 
However 10 is the actual common randomness shared between the two terminals in the model. Random variable Ki 
is used for the communication at round i. Then, the protocol proceeds as follows, 

• In the first round, knowing (b\, to, x"), terminal 1 generates a sequence /" according to P(f™\bi, u>,Xi) of 
protocol A, and sends the bin index fci(/f) of protocol A to the terminal 2. At the end of the first round, 
terminal 2 having (bi,ui, fci,^), uses the Slepian-Wolf decoder P (fi\b\, ki,ui,X2) of protocol A to obtain 
an estimate of /". We use /™ T to denote this estimate of /" by the second terminal T 2 . Since the first terminal 
knows /" we set /™ T = /" to be the estimate of /" by the first terminal Ti. 

• In the second round, knowing (&2,£2i/™t )> terminal 2 generates a sequence fg according to 
PF n \B 2 x n F n {f 2^ t x 2 t I it ) °f protocol A and sends the bin index A:2(/" T , f^j ) of Protocol A to the 
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terminal 1. At the end of the second round, terminal 1 having (b 2 , k2,x™, /™ T ), uses the 

^J^Tr k r« Y-nC/^I^S) fe, w,/" T , a;™) defined above to recover /^. We omit subscripts from the pmfs 

when they are clear from the context. 

• This procedure is repeated interactively for i S [3:7-]. Thus, at the end of the round r, the first terminal has 
fli 1 -r and the second terminal has f,™ , _ . 

•'[l:rJ,Ti ■'[l:r|,T 2 

• The first terminal uses the conditional distribution p{yx\x\, /[i :r ]) that we started with at the beginning to 
create y" from the conditional distribution p(j/™|x" , /r". r i T ) and the second terminal uses the conditional 
distribution p(y 2 |a; 2 ,/ [1:r] ) to create y£ fix>mp(y£|a#,/£ :r])Ta ). 

The random pmf induced by the protocol, denoted by P, factors as 

(Tl £Th L. 1 TL £T\j £Th \ 

X [l:2] if[l:r]> °[l'-r]> toll-r]iViV[l:2]T t[l:r],Ti> J[l-.r],T2 > 



~p{xl.. 2] )p u {u)p u {b {1 .. r] ) 



J] P(/r, fe|fc, C*. / [ " : ,- 1] ,T (l)2 , 4) 2 ) PW (/TT (1+1)2 |fc, *i, <*, /[l:i-l],T (i+1)2 , 4+1), 



(=1 



i{/iT Wa =/"} p(y?\f%r]^)p(y2\f% r] ,T^) 



— ^(#[1:2] 



]Ip ff (u;Op><)P(/P,k|&i,u;i,/k-i]^^ 

l{#T (4)a - /Dl P(»?l^] fTl «l )P(S/2 l/fl:,],^^)- (10) 



where Wi = u, and w, is a constant variable for i > 2. 

Parf (2) 0/ f/ze proof: Sufficient conditions that make the induced pmfs approximately the same: We need to find 
conditions that imply 

^ > ( a; fl:2]i/[l:r]i^[l:r])%:r])W,y[ 1:2 ],/[ 1:r ] jTl ,/[ 1 . r ] )T2 ) ~ P( x [l:2]> f[Ur]i &[l:r]j &[l:r]i ^>V[U2]' /[l:r],TV /[l:r],T 2 ) 

for some e„ converging to zero as n — > 00. We begin by proving that 

^( a [l:2]>/[l:r]>&[l:r])fe[l:r]>W, /[" :r ],Ti ! /[™:r],T 2 ) ~ -PC^fi^] ) /[™:r] ) ^[l:r]> %:r]> w > /[l:r],Ti> /[l:r],T a )' 

where we have dropped j/n. 2 i f rom t> om sides. We will add y 7 A. 2 ] to the equation later. 

To find the constraints that imply that the pmf P is close to the pmf P in total variation distance, we start with 
P and make it close to P in a few steps. For any j £ [0 : r] we inductively find constraints that imply 

P ( x [l:2]> /[ly]ify:j]>fyl:j]) W 3' /[lj],Ti ) /[l:j],T 2 ) ~ ^fl^] i /[i:j] i &[l:j] ! fc [l:j] i w j ! /[l:j],Ti i /[l:j],T 2 )• 

for some e n converging to zero as n — > 00, where w\ = (J, and ljj is a constant variable for j > 2. For j = 
this is trivial since it reduces to -P(^m. 2 i) = P( x 7i-2\) ~ P( x Ti-2])' We snow m Appendix lAl that the constraints 
sufficient to guarantee the statement for j given that it holds for j — 1 are as follows: 
1) Reliability of SW decoders: For j = 1 the SW decoding is reliable if, 

R 1 +Ro + Ri>H(F 1 \X 2 ). (11) 
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For j > 2 the SW decoding is reliable if, 

V*G[2:r]: Ri + Ri > HiF^+^F^^). (12) 

2) Other constraints: For j = 1 we have the constraint 

Rq + Rx<H{Fi\Xi). (13) 

For j ' > 2 we have the constraints 

Rt < HiFilX^F^^), for i = 2, • • • ,r. (14) 

The details can be found in Appendix [A] but a brief description is in order. Comparing equations ((9) and (Hot 
we see that most of the terms are the same if we assume that the Slepian-Wolf decoders succeed with probability 
one. Constraints ( fTTT i and ( fT2l guarantee the success of the Slepian-Wolf decoders with high probability. To make 
the two pmfs close we need to guarantee that -P(6j, Wi|/r™. i _ 1 i T , x?^ ) « p u (uJi)p u (bi). In other words we need 
constraints ensuring the uniformity of (6,-,w,-) and its independence of (/n.^n T , #K-\ )■ These constraints are 
given in equations (\3[ and (TT4l i. and are obtained using Theorem [3] 

Therefore equations ©, ([Tol l. (fT3l and ([Pit imply that 

-f^fl^] i /[" r] i &[l:r] j %:r] i W, /[l:r],Ti i /[l:r],T 2 ) ~ ^(^[1:2] i /[l:r] ' &[l:r] ! fyl:r] ) w i /[l :r ] ,Ti ' ^[l:r] ,T 2 )- 

In Appendix [B] we show that the equations (O, dTOb imply that 

"t I [l:2]i/[l:r]!' ) [l:r]i' i [l:r]! W ' ^[1:2]> /[l:r],Ti > / [l:r],T 2 J W "^[1:2] > /[l:r]> & [l:r]) %l:r]> W ) S/[l:2]> /[l:r],Ti > / [l:r],T 2 /' 

Using part one of Lemma [2] we can deduce the same approximation over the marginals 

■P(6[l:r],a"i:2].l/p:2]) « ^(&[M> ^1:2]. 2/fl:2]) (15) 

for some e„ converging to zero as n — > 00. In particular, the marginal pmf of (^n^i^i^i) °f me RHS of this 
expression is equal to p(ajK. 2 i,yR. 2 i) which is the desired pmf. 

Part (3J o/ the proof: Eliminating the shared randomness: 

In the protocol we assumed that the terminals have access to shared randomness Bu._ r ] which is not present in 
the model (note that u is the real common randomness shared between the two terminals in the model). To get rid 
of the shared randomness S[i :r i, we would like to condition on a particular instance of Sn. r i = bn :r -\. However, 
conditioning on b\i :r ] may change the marginal pmf of -XTJ.jij ^r™2i on tne ^HS °f 03- Thus, we want to impose 
certain constraints on the size of the bins to guarantee that the marginal pmfs do not change. In other words the 
induced pmf P(xK. 21 , J/n. 2 i) changes to the conditional pmf P(x? 1 ^,yV 1 . 2 -Xb\ i i. r ^). But if Sfi :r i is independent of 
(XS.j,, ^r".2i)> men me conditional pmf P(xm. 2 i, J/n-21 ltyi:rl) i s a l so c l° se to me desired distribution. Therefore we 
can assume that the terminals agree on an instance b\\- r ] of Bu-r] and run protocol B. More precisely, to obtain the 
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independence, we use Theorem [3] where we substitute T = r, Xi = -Fliti] and Y — Xn^iYri^i t0 g et the following 
sufficient condition for the pmf of protocol A: 

i 

V»e[l:r], Y,Rt<H(F [1:i] \X [1:2] Y [1:2] ). (16) 

t=i 

This implies that 

P Q>[l*-]> X [ia]'Vlia]) ^ P U ( h ^-r\)P( X [l:2YV[l:2])- ( 17 ) 

Equations < fT3T > and (fTTJ in conjunction with the third part of Lemma [2] imply that 

P(k [1:r] ,2f 1:2] ,2/f 1:2] ) "*» " p U (b [v . r] )p(x^. 2] ,yf 1 . 2] ). (18) 

Using Definition [4] equation ( fT8l guarantees existence of a fixed binning with the corresponding pmf p such that 
if we replace P with p in (O and denote the resulting pmf with p. This would then imply that 

P( fo [l:r],2fi:2];2/[1:2]) ^ " P U (&[l:r])p(*fl:2]> #[1:2])' 

Now, the second part of Lemma [2] shows that there exists an instance 6r 1:r i such that 

P( X [l:2]^y[l:2]\ b [l:r]) ~ P(*[l:2]^[l:2])' 

We have found all the necessary constraints on the size of the bins for protocol to work. Finally, eliminating 
(i?i, • • ■ ,R r ) from the inequalities ([TTIi-([T4li and ([Tot gives rise to the constraints given in the statement of the 
problem. This completes the proof of the achievability of Theorem Q] 

C. Achievability proof of Theorem [2] 

The achievability comes from setting i?o = oo in Theorem Q] 

V. Converse 

Let (i?o, Ri2, R21) be an achievable rate tuple for r rounds of communications. Then, for any e < |, there exists 
a simulation code of length n such that the total variation between the induced pmf p{y r A.2\i x ?i .21) anc ^ tn e n i.i.d. 
repetitions of the desired pmf q(x,y) is less than e. 

A. Mutual information bounds 

The following lemmas which are consequences of a generalized version of Lemma 2.7 of (9), will be useful 
throughout the proof of the converse. The proofs are provided in the Appendix ICl 

Lemma 3 For any discrete random variables W n and Z whose joint pmf satisfies 

1 



p(w n ,z) -p(z)[[p q (w q \z) 

9=1 



<£< 4' 
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for some p q (w\z), we have 



9=1 



Lemma 4 Take an arbitrary i.i.d. sequence X n distributed according to p(x) and a conditional pmf p(y n \x n ) 
which is not necessarily i.i.d. If there exists a conditional pmf p{y\x) such that 



Ky n K)l[p(x q )-l[p(x q )p(u 

9=1 9 =1 



then 



where [~ q] = [1 : n]\{g}. 



q^q, 



\y\ 



<£< 4' 



I{X^ q] -Y q \Xq)<Ae\og^, 



B. Proof of converse 

Take a random variable Q uniform on [1 : n] and independent of all other random variables. Define Fi — 
wCiX^ n X 2 ~ Q for 1 < i < r and Xi = XiQ, Y{ = Yiq for i = 1, 2. In the first step of the proof, we show 
the Markov chain conditions given in the definition of T(r) are satisfied by this choice of auxiliary r.v.'s. These 
conditions are equivalent with the following 

Ci—cdC[i;i^i]Xf' n X 2 ' q ~ — X2. q if i is odd, 
Ci— u)Cii-i_i]Xf ' n X 2 q — Xi tq if i is even, 

Y\^ q —UjC[l :r \X 1 X 2 — Xl,qYn,qi 

Y2,q— UjC[l- r ]Xi ' n X 2 q — Xl^qYl^q. 

The proof is provided in Appendix [D] 
We know that 

P( X [1:2]' 3/[l:2]) _ 1\ X [1:2] ' 2/[l:2]) 

where p(xn . 2 i , 2/m . 2 i) is the induced pmf of the code. This implies that for any value of Q = q, 

\\p(X[U2],q,y[l:2],q) ~ q{%[\:2] , 2/[l:2]) || j < C, 

therefore the total variation distance between the average of p{X[i:2],q>y[i:2],q) over Q — 1 (i- e - P( x [i-.2],q, V[i-.2],q)) 
and q(x\i : 2],y[i:2]) is small. Next we have 

nR 12 > J2 H{d) 



<e, 



i:odd 



i:odd 
r 

= ^7(C i ;X 1 "|q 1:i _ 1] X«u;) 



(19) 
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= /( W q 1:r] ;X 1 "|X 2 ") (20) 

n 

q=\ 
n 

= J2 1(uC [1:r] Xl +v - n X 2 ^ q ; X hq \X 2 , q ) (21) 

g=l 

n 

>Y,I^C {1 .. r] X^ 1 -- n Xl-' l - 1 ;X x>q \X^ q ) 

9=1 

= fir(«q 1 . r] X? +1:n ^ !Q - 1 ;A' 1 ,o|X2,o,Q) (22) 

= n/(wq 1:r] X 1 Q+1:n X 2 1:Q - 1 Q; Xi, Q |X 2 , Q ) (23) 

= n/( J F [1:r] ;X 1 |X 2 ) (24) 

where ( fT9b follows from the Markov chain Ci — C^.j-^X^w — X™ for even i, J20l > is due to the independence 
of common randomness u> from X™X 2 l and the rest of the equations follow from the fact that X\ q ,X2 q are i.i.d. 
repetitions. 

A similar statement can be proved for i? 2 i: 

^2i>/(^[i: r ];X 2 |X 1 ). (25) 

Next consider, 



n(R 12 + R )>H(uC 1 )+ J2 H( - C ^ 

i:odd, z>l 

>#(u;Ci|X 2 ") + ]T fl r (Ci|q l!i _ 1] Jf a n w) 

i:odd, z>l 

>/( w c i; y [ ? :2] x 1 ™|x 2 1 )+ £ /(c^xnqx^rjXjw) 

i:odd, i>l 

= J(wC i; l^ :2] |.Xfl !2] ) +7(u;C 1 ;X 1 "|X«) + ^7(C i ;X 1 "|q 1:i _ 1] X« W ) (26) 

= J(wCi ; Y[? :2] |X[™ 2] ) + J(wCi ; X? |X 2 ") + 7(C [2:r] ; X? |X 2 n C lW ) 
=7(u;C 1 ;y [ ? :2] |X^ 2] ) + /(q 1:r] a;;X 1 ™|X 2 ") 
> /(wC^y^lX^,) +n/(F [1:r] ;X 1 |X 2 ), (27) 

where (l26l i follows from the Markov chain C; — Cu^-ijX^ui — X™ for even i. Equation d27l i follows equality of 
equations d20b and ( 1241 ). Now, we work out the first term of equation ( l27l . 

n 

I ( UjC ^ Y [l:2]\ X \l:2]) = 2^,^{ uC ^ Y [l,%<l\ X [X:2Y Y [ia] ) 
9=1 

n n 

= z2 I (uClY [i q 2] ;Y[ ll2 ] iq \Xp_. 2] ) -22H Y [li} '> Y [^U\ X [l:2\) 
9 =1 q=\ 
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(a) 



> ^/(wCi^^lX^-ruKe) 



3=1 



= z2 J Kl^[l:2],-9; ^il:2], g |^"[l:2],g) - 2_/ ■^(•^'[1:2],~?; *fl:2],g|-X'[l.:2],g) ~ "#00 
g=l g=l 

/, v n 

> ^/( W C 1 ^ +1: "X 2 1:? - 1 ;y [1:2] , 9 |X [1:2hg ) - 2n 5 ( £ ) 

= n/( W CiX 1 Q+1:n X 2 1:Q - 1 ;y [1:2] , Q |X [1:2]j Q,Q) - 2n 5 (e) 

= nI(QwC 1 X 1 Q+1: "X 2 1:Q - 1 ; ^ 1:2] ,q|X [1:2] ,q) - n7(Q; 11i: 2 ],q!^[i : 2],q) - 2ng(e) 

>n7(F i; y [l!2] |X [1:2] )-3ns(e) (28) 

where 5(e) := 4elog( Ei2Ui [_£!] ) ; ( a ) i s a result of Lemma[3] (b) follows from the Lemma|4]and the last inequality 
is a result of (8] Lemma 21]. 

Equations (J27j and ( |28l > imply that 

i?12+i?0 >/(Fi;y[l:2] |X[l:2])+/(i ? [l:r ] ;Xl |^ 2 ) -3. 9 (e). (29) 

Following the same lines as in the previous cases, we can show that 
n(Ro + R 12 + R21) > HiwC^X^) + £ Hid^C^^X?) + £ H(C i \wC [l:i _ 1] X?) 

i>l, i'.odd iieven 

> I{wC 1 iY [ l 2] X?\X2)+ J2 I(Ci-,Y[ia ] Xi\uC llsi _ 1] X?)+ £ 7(Q;y [ ? :2] X 2 >q 1:i _ 1] jq i ) 

i-.odd i:even 

= /( w c i; xr|x 2 ") + j( w c l5 y£ 2] |xp 1:2] ) 

+ ^ [7(C i ;X 1 »|a;C [w _ 1] X 2 ™) + 7(C i ;yf :2] | W q 1:i _ 1] Xp i:2] )" 

i-.odd 
i>l 

+ £ [7(C i ;X 2 "|a;C [1:i _ 1] Xr)+7(a;yf :2] |a;q 1:i _ 1] X [ ™ ;2] )" 

Ziehen 

= /(cC 1 ;X 1 "|X 2 l )+ ^/(C l ;X 1 "| w q 1:J _ 1] X 2 l )+ ^ 7(a-;X 2 >q 1:i _ 1] Xf) 

■i:odd i:even 

i>l 

+ 7( w c i; y [ j 2] |x [ « :2] )+ 2 7(c i; yf :2] |u;C' [1:i _ 1] x^ 2] )+ £ ^y^q^xfi^) 

*:o^d i:even 

i>l 

= I(uCv,X?\X2)+Y,I(Ci;X?\uCli:i-i}X2)+ E 7(C , i ;X 2 >C [1:i _ 1] X 1 n ) 

i>l 

+ /( w c i; y [ 5 l :2] |x [ » :2] ) + E/(c l ;y [ j 2] | w q 1:l _ 1] x [ " 1:2] ) 

i>l 

= 7( W C 1 ;Xr|X 2 ") + E7(Q;X 1 ™|u;q 1:i _ 1] X 2 n ) + ^7(C i ;X 2 "|u;q 1:i _ 1] X 1 ") 



j>i 



+ 7(wq i:r ]; y[™ 2 ]|-^[™:2]) 

7( w q 1:r] ;X 1 "|X 2 ") + 7(u;q 1:r] ;X 2 "|Xf)+7( w q 1:r . ] ;yf :2] |X [ " 1:2] ) 
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>n(7(F [1:r] ;X li Q|X 2 ,Q)+/( J P [1:r] ;X 2i Q|X 1)Q ) + 7(F [1:r] ;y [1:2LQ |X [1:2] ,Q)-3 5 (e„)) 

(30) 
where the first term of (l30t follows from equality of equations d20l i and d24l i. second term follows similarly and 
the last term follows from an argument similar to the one given in deriving equation d28T i. 

In summary, we have proved that for every e < |, any achievable rate tuple must belong to the set S t (r) defined 
as the set of all tuples (i? ,i?i 2 ,i?2i) such that there exists p(/n :r ],a;ri :2 ], J/[i:2l) G ^e( r ) f° r which (R , R12 , R21) 
satisfies the inequalities d24t . (1251 1. (|29l i and (13 Oi l, where T c (r) is the set of p(/[i :r .i,a;ri :2 ], j/ri :2 i) satisfying the 
Markov relations as in the definition of T(r) and 

\\p(x[l:2],y[l:2]) ~ q{X[l:2] , 2/[l:2]) || 1 < £■ 

The proof continues by showing that n c >oS e (r) = S(r). 

C. Converse Proof of Theorem \2\ 

Assume (i?i 2 ,i? 2 i) is a pair of achievable rate. Consider a sequence of coordination codes that achieves 
(i?i 2 ,i? 2 i). Take a random variable Q uniform on [1 : n] and independent of all other random variables. Define 



Ft = aX 1 Q+1:n X 2 1:Q ~ 1 Q for 1 < i < r and X % = X lQ .Yi = Y iQ for i = 1, ^l In the first step of the proof, we 
show the Markov chain conditions given in the definition of T(r) are satisfied by this choice of auxiliary r.v.'s. The 
proof of this fact is similar to the one given in the Appendix [D] and hence it is omitted here. 



nR l2 > J2 H ( C >) 

i:odd 

i:odd 
r 

= ^J(C i ;X 1 "|q 1:i _ 1] X 2 ") (31) 

4=1 

= I(C[l:r]',Xl\X£) 

n 
= / , HC\l:r}'i Xl, q \Xl '"^2") 
9=1 
n 

= Y, I(C [1:r] Xf +1 -- n X2^ q ; X ltq \X 2>q ) (32) 

q=l 
n 

>Y,mi--r]Xl +1:n x l 2 q - 1 ;Xi, q \X2, q ) 

9=1 

= nI(C [1:r] Xf +1:n X^ Q - 1 ;X hQ \X2, Q ,Q) 

= n/(C [1:r] X 1 Q+1:n X 2 1:Q - 1 Q; X 1>Q \X 2>Q ) (33) 

Following the standard definition of empirical coordination code we assume that there is not any common randomness, that is u) is a constant 



random variable. See |8| and Remark [4] 
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= nI(F [1 .. r] ;X 1 \X 2 ), (34) 

where (|3T1 i follows from the Markov chain C{ — Cti-i^-nX^ X™ for even i, d32l > follows from the fact that X\ q , X2 q 
are i.i.d. repetitions and ( l33l > follows from the fact that Q is independent of (X^q, X 2 ,q) (See |8|). The inequality 
-R21 > ■f(-P[i:r])-^'2|-X'i) can be proved similarly. 
The definition of coordination code implies that 



E 
This yields that 



PXJ] :2] Y,^, ~ ffXn.a.YM,™ 



-» 0. (35) 

1 



V X ?X*] Y { 1. 2] ^ 1X [1:2] Y [1:2 y (36) 

In the other side, it is shown in |J8] that Epx™ Y n = PXp.,, Q ,Yfi. 2 ] Q where ft is the induced pmf by the code. 
Therefore ftx [x . 2] Q ,Yn , 2] Q tends to gxri . 2] Yji 21 ■ Now the closedness of the coordination rate region completes the 
proof. 
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Appendix A 
In this appendix we find the constraints that imply 

-^"(^[1:2] > f[l:r] > &[l:r] ) fylrr] ) w [l:r] > /[":r],Ti ' /[":r],T 2 ) ~ ^(^[l^] > f[l:r] > &[l:r] > fyltr] ) w [l:r] t /[™:r],Ti ' /[™:r],T 2 ) • 

Let Z = X™. 2] and Z. } = {Fl>,Bj,Kj, Wj,Fg Tl ,F£ T2 ) for j e [1 : r}. For any j e [0 : r] we inductively find 
constraints that imply 

P(Z [1:j] ) f P(Z [1:j] ), (37) 
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^ (i) 

for some e„ ' converging to zero as n — > oc. 

Let us define a new random pmf P by changing one of the terms in the expansion of the pmf P of the protocol 

A given in (|9]). We replace the Slepian-Wolf terms with one that corresponds to an ideal zero probability of error. 

P\ x [l:2] >/[l:r] ' fyl:r] i &[l:r] i w j V[l:2] > /[™:r],Ti > /[l:r],T a ) 



P«l:2l) 



Ij p(6„ w*!^.!], xf l)2 )p(/r, Aiihi, Wi , /^i],^) 



2=1 



1{/" T(!+1)2 = /T>l{/"T Wa = /"}J jKtfl/frr]*?MV?l/[l:r]S2)- (38) 

In order to show that the pmfs P and P in (f3Tb are close, we show that both are close to P. Therefore they have 
to be also close to each other because of the triangle inequality. In other words we will inductively find constraints 
that imply 

P(Z [1:k] )% P(Z [uk] ), forfce[0:r], 



P(Z [1:k] ) & P(Z [1:k] ), for fc € [0 : r], 



(39) 



Jfc) 



for some e K n' converging to zero as n — > oo. For j = this is trivial since it reduces to P(in. 2 ,) = p(#h. 2 i) = 
P{x? x . 2 t) = P(xl l l . 2 i). Suppose that d39l holds for k = j — 1. To show it for k = j we proceed as follows. First 
observe that it suffices to prove the existence of a sequence S n — ► such that 

P{Z m ) = P(Z [0:J _ 1] )P(Z J |Z [0:J _ 1] ) fe P(Z [1:J _ 1] )P(Z J |Z [0:J _ 1] ), (40a) 

P(Z m ) = P(Z [0:J _ 1] )P(Z J |Z [0:J _ 1] ) fe P(Z [0:J _ 1] )P(Z,|Z [0:J _ 1] ), (40b) 

because the third part of Lemma [2] then yields that 

p{z m ) e i ?{z m ), 
P(z [0:j] ) £ i P(z [0:j] ), 

where e« = e« +5 n . Next, note that the triangle inequality implies that instead of showing ( I40bb one can show 



s n 



40a| ) and ( 140ct given below 



P(% :j -_ 1] )P(^|Z [0:i _i ] ) S P(Z [0:J _ 1] )P(Z J |Z [0:J _ 1] ). (40c) 

Therefore it suffices to show (I40ab and (I40ct . 

We begin by finding the expressions for the terms appearing in J40l >. The marginal pmf P(Z[ 0:J _ 1 ]) (computed 
from equation d38l ) is as follows: 



P(Z [0:j _ 1] )=p(xf 1:2] ) 



P(X?1:2 



'i-i 



j=l 

xi{^ T(4+1)a =/r}i{^T Wa = y?} 



(41) 



j-i 



Il P (/"l/[l:i-l]' a; ri) 2 ) P ( 6 » a; » A: 'l/[l:i]'4) 



Februaiy 27, 2013 



P(*ri:2 



J-l 



xi{^ T(i+1)2 =ni{^T W2 = /"} 

IIP(/ri/[W-l]^W a )^.W<.*il/[l:<]) 

xi{/" T(I+1)2 =/r}i{i?:T (s)2 = /r} 



(42) 



P( X fl:2]'/[":j-l]) 



J-1 



n p (6i.w*.*ii/iw 



i=l 



(43) 



xl{^ (!+1)2 =/I l }l{/rr (s)2 =/T} , 

where equation d42l follows from the fact that (&i, w», hi) are (random) bin indices of /£.* in Protocol A, and that 
(FT 1 , Fm . i _ 1 -i , X^-, ) have an i.i.d. pmf in the same protocol. Equation (l43l follows from the Markov conditions on 

*[":2].*i™i-l] inT M- 

There are three conditional pmfs in (140} that can be computed from equations (l38l l. (l23l . (l24l respectively as 
follows: 

p(^|% i -i]) = p(6 i ,« i |/ I Vi]^a)J p (>7.*il 6 i. w i./[Vi]' a! w)J 

x !{^t (3+1)2 - mnf-T U)2 - m> < 44 > 

?(z^Vii)= p fe^ii/[ii-i]-^J p (/;^^i."i-/[ii-i].^) 

x^^J^^^/^^U) 1 ^^''- < 45 > 

^^(^J^^'^ril^/U) 1 ^,,^!- < 46 > 

Finding sufficient conditions for equation ( 140al l to /ioW: 

We begin by showing equation J40al >. Note that the only difference in the two pmf expressions is that the Slepian- 
Wolf term in (|45]> is replaced with an indicator function in (l44l . To use Slepian-Wolf theorem we need to show that 
we are dealing with an i.i.d. scenario where random bin indices are transmitted from one party to another party. 
Let us rewrite equations (l45t and (l44l as follows: 

P^-lZ^j) = P(/ i n |/ [ ^_ 1] ,x^ )2 )P(6 J -,u; J -,fc i |/5 !jl ,x^ )a ) 
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We now compute P(Zr Q .j_ii)P(Zj\Zt .j_i]) and P(Zr Q .j_ii)P(Zj\Zt .j_i]) using equation d43l as follows: 



P(Zp :j _ 1] )P(Z J \Z [0 .j_ 1] )=p(xl i2]) fJl SJ < i ) 



i-i 



npft.^kii^)!^ =/p}i{# T(0a =/n 



x%«i,k»IW{ft Tu)l =^} 



(49) 



(50) 



P(^[0:i-l])P(^|^[0:i-l])=p(^: a ],/5: J -]) 



i-i 



I]Pft )Wi! M/ [ l, 1 )i{^T ti+1)2 =/i}i{4, 2 =/"} 



^(At^i^^^^W (51) 

Using the first part of Lemma [2] it suffices to show that 

The above pmf corresponds to an Slepian-Wolf problem where the first party has i.i.d. repetitions (/,". ._.,, , x?-. ) and 
the second party has i.i.d. repetitions (/S ._ x , , a^.jn )■ The first party creates i.i.d. repetitions /" and communicates 
random bin indices bj,kj,UJj of /£ ., to the second party. Using Lemma Q] the above total variation is small as 
long as the following constraints hold: 

• For j = 1, loj is non-empty and the SW decoding is reliable if, 



Ri+Rq + Ri >H{F 1 \X 2 ). 
• For j > 2 the SW decoding is reliable if, 

R j + Rj > HiF^X^^F^^) = H^IXy+D^py.!] 

Finding sufficient conditions for equation ( |40c| l to /zoW: 



(52) 



(53) 



The pmf P(Z[ 0: j_i])P(2j|Z[ 0: j_i]) was computed in equation ( |5TT l. We now compute P(Z[ a .j_ 1 ])P(Zj\Z\ Q .j_ 1 ]) 
using equations (l43l and d46i i as follows: 



P(Z [0y -l])P(^|Z [0;i _ 1] )=p( a: f 1:2] , /[!,■_!]) 



J-1 



.4=1 

xp cr ( Wj -)p I '(^)i > (/;.* J -|6i,« i> /^_ 1]iTo)s ,x^ )a )i{^ Tw)a - /;} 



P( X [l:2]'/[™:j-l]) 



J-l 



npift^i.fei/^])!!^,, = /ni{/" TW2 = /n 



xp a ( Wj op a (^)p(/;,fc J ^,-,^,/ [ v 1 ]^o) 2 )i{/j:T W)2 =/;} 

x P (^T w+I)2 l&i)fej>Wj,/[i.j-_i],a;^ +1)a ) 



(54) 
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P(*|i:2]>/[ly-l] 

xp tr (w i )p^(6 i )P(/?|6 i) w i ,^ ! ,._ 1] , a; ^ a )P(fc i |^ !i] )l{^ Twa =i7} 



x pSW (/i!T - +1)2 l^^i>^>/[Vi].^ + i) 2 )' (55) 

where (|54] > holds since P™. _ 2 , = Fft. _ ± , holds because of the indicator functions in P(£[o:j-i])> equation d55l ) 
holds since fej is a (random) bin index of /r™. 7 i- 

Let us compare (|55T > and (IBTl l. We see that most of the terms are the same. Using the first part of Lemma [2] it 
suffices to show that 

p(xf 1:2] ,^ :J _ 1] )^K)^(6 J )^(/;l^,^, /[":,-!]> ^- )2 ). (56) 

Note that 

K4:2],/[Vl]M/"kfl :2 ]>/ [ " :j -l])m,^l/[": J -]) = 
P^l.^-llJ^^.Wil^i-l],*"!^)^^!^.^.^-!].^)- ( 57 > 

We note that P(/ n |&j,u;j, /r" -_ 1 i , ^7,) ) of the above equation is the one of Protocol A and used in Protocol B. 
Now, to show that d56i l holds it suffices to show the following equation because the first part of Lemma [2] 

P(*[i:2], /[Vl] W^l/[l:i-l] ^[1:2]) « P(^l:2] ./[":,-!] ^(^ fe ) ■ (58) 

In other words we need to impose constraints that imply (Bj , u>j ) are mutually nearly independent of (Pn .,-_]i , XT\ . 2 ,). 
Substituting T = 1, Xi = Pn : j] and y = X[i : 2]P[i : j_i] in Theorem [3] yields that equation (I5"8l holds if 

• For j = 1 we have the constraint 

Po + Ri < H(F 1 \X 1 X 2 ) = PCPilXx), (59) 

• for j > 2 we have the constraints 

Ri < HiFjlX^F^^) = HiF.lX^F^^), (60) 

where in ( |59l and d60l > we use the Markov chain Fj — X(j) 3 F\i:j—i\ — ^(j+i) 2 f° r an Y J- 
This completes the induction proof. 

Appendix B 
In this appendix we show that the approximation 

- P ( X [l:2]'/['l:r]' & [l:r],fc[l:r] ) w [l:r],/[ ? l: r ] ! T 1 '/['l:r],T2) ~ ^(' T fl:2] ' /[":r] '^[l:r] , fc [l:r] , w [l:r] , /[™ :r ] ^j ' /[":r] ,T 2 )' ( 61 ) 

implies 

P( ;r [l:2]'/[ , l:r]^[l:r-]ifc[l:r],^,yfi:2]:/[l:r],Ti'^[ ? l:»-],T2) ~ ^l^fia] ! /[l:r] > ^[I.t] ' ^[l:r] : w : !/fl:2] > /[l:r],Ti> /[l: r],T 2 )' 

(62) 
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for some sequence e„ — > 0. 

We prove it indirectly through the random pmf P introduced in (138V It has been shown in Appendix [A] that in 
addition to the approximation doTl i the following approximation holds 

P ( x [l:2]J{l:r]'^r]' k {l:r]^[i; r ]J{[ :r] j 1 J{ l 1:r] j 2 ) « P(xf l:2] , /£.,.], i»[ 1:r ], k[u r ] , W[i :r ], /[l :r j jTl . /[™:r],T 2 )- (63) 



Note that the triangle inequality implies that instead of showing the approximation (I62l i one can show the following 
approximations 

P( x [i:2] ) /[l:r] ' &[l:r] J tyltr] 7 W, y[^. 2 ] , /[l:r],Ti 1 /[":r],T a ) ~ ^(^[l^] > /pL:r] ' &[l:r] ) fyl:r] ) w > 2/[l:2] 1 /[i:r] ,Ti ' /[l:r] ,T 2 )> 

■Pfcjira] ) /[™:r] 1 &[l:r] ) %:r] 7 w , Z/[i : 2] > /[l:r],Ti J /[™:r],T 2 ) ~ P\ x \l:2\ > /[™:r] ) &[l:r] : &[l:r] 1 W ! S/[i:2] 1 /[™:r] ,Ti ' fjl:r] ,T 2 )' 

(64) 

Using the third part of Lemma [2] it suffices to prove the following approximations 

P( X [L:2] > /[™:r] ) ^[l:r]i fylrr] j w j /[l:r],Ti ' /[i:r],T a )-P(#[i:2] l x [i:2] > Z[":r] > &[l:r] J ^[l:r]; w ; /[™ r ],Ti ' /[™:r],T 2 ) 
~ P( x fl:2]>/[™:r]>fyl:r]>%:r]>W,J/^. 2 ], /["^j^) /[":r],T 2 )> 

■P( a; p.:2] ' /[l:r] ' &[l:r] ) fyl:r] i w j /[l:r],Ti ' /[" r] ,T 2 )P(2/[i:2] F[i:2] i /[l:r] ' &[l:r] i fyl:r] 7 W 7 /[l:r],Ti ' /[l:r],T 2 ) 

^ ^(^^'/[l^i^ltrl^llrrJ.W.y^.a], /[l:r],Tii /[l:r],T 2 )- ( 65 ) 

First observe that 

^(y[l:2]Ffl:2]>/[™:r]'^[l:>-]>%:r])W,/[" :r ] ]Tl ,/[" :T .] )T2 ) = P (j/[i;2] F[i:2]> /[™:r]> &[l:r]> tyl:»-]> W > /[i:r],Ti> /[™:r],T 2 ) 

= P(»?l^]*l)P(tfal/p:r]*a)- 

This equation gives the first approximation of ((65) with equality. 
Next using equation (TToT > we get 

nyil^wJ^Altrb^r]^ (66) 

Substituting this in the second equation of (RxBT l gives thesecond approximation of (f65t with equality as follows 

-P( x [i:2] ! /[l:r] ' k[l:r] i fc[l:r] J w j /[l:r],Ti ) /[br] ,T 2 )P(2/[1:2] F[i:2] > /[l:r] ' ^[1^] J %:r] 7 W, /["^n i /[l:r],T 2 ) 

= ^(*ri:2].^:r].6[l:r],V]' W ^:r],Tx.^:r],T 8 )P(»"l/p:r]«"Mtfal/p:r]*5) (67) 

= P(*fl:2]»/[l:r]) 6 [l:r].%:r])W,J/[l :2 ], /[" r]]Tl , /[i :r ],T 2 )- (68) 

where the equation j67] i is due to the equality /r™. rl T = /n. r i j = /n- r i which is a result of indicator functions 
in the definition of P in d38l . This completes the proof of the approximation 
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Appendix C 



A. Generalized version of Lemma 2. 7 of ^2l? 



Lemma 5 ([9|) For any two pmfs px and p x on the same alphabet X, we have 

\X\ 



H(X) - H(X) 



< 2\\px -p^l^log 



2||px -PjtWi 



(69) 



We now state a conditional extension of this lemma. 



Lemma 6 For any pmfpy on y and any two conditional pmfs Px\Y and p^, Y on the same alphabet X such that 



we have 



Proof: 
H{X\Y) -H{X\Y)\ 



PYPX\Y -PYP X \Y 

H(X\Y) -H(X\Y) 



\y una Pv\y 
1 



i SeS 4- 



**<• 



(70) 



J2py(v) (h{X\Y = y)- H(X\Y = y) 
y 

< J2py(v) \ h ( x \ y = y)- H ( X \ Y = y) 

V 

< ^2py(v) x 2 

v 

< (25>y(2/) 
V v 



PX\Y=y-P X \Y=y 



log 

1 » 2 



\X\ 



PX\Y=y ~ P X \Y= % 



PX\Y=y ~ P X \Y=y 

\x\ 



log 
!/ ^E y PY(y) 



PX\Y=y -P X \Y=y 



(71) 



(72) 



PYPX\Y -PYP X \Y 



log 



\X\ 



PyPx\y ~PyP x \y 



\X\ 
<2 £ logLi, 



(73) 



where ( |7T| > follows from Lemma |5] ( |72t follows from Jensen inequality for the concave function g(t) — t\og L^ 
and ( 1731 foliows from the fact that g(t) is an increasing function on [0, i], ■ 

B. Proof of Lemma \3\ 

The proof is similar to the one given in [8. Lemma 20] for the unconditional case of the Lemma [3] First we use 
the first part of Lemma [2] to obtain the closeness of p(w q , z) and p q (w q \z)p(z) in total variation. In other words 
we have 

\\p(w q ,z)-p q (w q \z)p(z)\\ 1 < e. 

Let W n be a random variable such that p^ n z (w n ,z) — p{z)Y\ q=1 p q (w q \z). Then Lemma |6] implies that 

w 



H{W q \Z)-H{W q \Z) 



< 2e log • 



2e ' 



q = 1,2,--- ,n, 
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H(W n \Z)-H(W n \Z) 



„ , |W|" „ , |>V| 

<2elog^^ = 2nelog^. 



(74) 



Now we have 



Y J I{W q] W"' 1 \Z)=Y J H{W q \Z)-H{W n \Z) 

9=1 9 =1 



J2[H(W q \Z) - H(W q \Z) 
Wl 



9=1 

< Ane log 



ff(W n |Z)-#(W"|Z) 



2c 



(75) 



(76) 



where ([75) follows from the fact that iT(W™ | Z) = £™ =1 H(W q \Z), because of p^„ z (w", z) = p{z)J\^ =1 p q {w q \z) 
and ((76) is a result of ( 174) . This concludes the proof. 

C. Proof of Lemma [4] 

The proof is similar to the proof of Lemma [3] First, using the first part of Lemma [2] we have 

\\p{y q ,x q ) -p(y q \x q )p{x q )\\ 1 < e. 

Let Y n be a random variable such that p x „ y„(x n , y n ) — Y\^, = iP{x q )p(y n \x n ). Observe that H(Y q \X n ) = 
H(Y q \X q ), Then Lemma |6] implies that 

\y\ 



We have 



H(Y q \X q ) - H(Y q \X q ) 

H(Y q \X n ) - H(Y q \X n ) 



I(X^ q] ;Y q \X q ) = H{Y q \X q ) - H(Y q \X n ) 



< 2elog 

< 2e log 



2e' 

2e ' 



(77) 



< 



H(Y q \X q ) - H{Y q \X q ) + H(Y q \X q ) - H{Y q \X' 
H(Y q \X q ) - H(Y q \X q ) + H{Y q \X n ) - H{Y q \X n ) 



< 4e log 



2c' 



(78) 



where (|78) follows from (177) . This concludes the proof. 

Appendix D 
We know that for any code the following Markov chain conditions hold 

Ci-u>C[ M _i]Xf - X$, if i is odd, 

Ci-wC[i :4 _i]X2 l - X", if i is even, 
Y x — tuC[i :r ]X 1 — X 2 Y 2 , 



(79) 
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The Markov chain Ci — wC[ 1:i _ 1 ]X]''"X 2 ' 9 ~ — X 2 , q for odd i holds because 

= IiXl^-X^uC^^XrX 1 ^- 1 ) (80) 

= 0, (81) 

where < T80b follows from the first Markov chain of ((79}, and the (ISTl i follows from Lemma |7] provided at the end 
of this appendix. Similarly the Markov chain Ci — wCriti-ii-X'i '™X 2 ' 9 — -^i,g f° r even * holds. Next, we show 
that the Markov chain Y\. q — wC[ 1:r ]I 1 , '™I 2 ' r — X 2q Y 2q holds. 

I(Yl,q'fX2,qY2 J q\ojC[i :r ]X 1 ' X 2 )< I(Yx j qX 1 ' ; X2,gl2,g|wC[i. r ]X 1 ° X 2 ) 

= I(X 1 1 :q - 1 ;X 2tq Y 2t q\ U C [1:r] Xf- n X^ q - 1 ) (82) 

< /(x^- 1 ; xf : "Y 2 , g | w q 1:r] xf n x l 2 q - 1 ) 

= I{X 1 { q - 1 ;Xf n \uC [l , r] Xf n X^ q - x ) (83) 

= 0, (84) 

where (|82l follows from the third Markov chain of (179} . (183} follows from the last Markov chain of ( |79l and the 
(l84l follows from Lemma |7] Similarly the Markov chain Y 2 , q — wC[i :r ]Ij ' n X 2 q — Xi tq Y\ tq holds. 

Lemma 7 For any set of random variables satisfying the Markov chain constraints of M9\ , the following holds: 

Vq,i: I(X 1 1 - q - 1 ] X q:n \ujC [l .. i] XrX^ q - 1 ) = 0. (85) 

Proof: We prove the lemma by induction on i. For i = 0, we have I(X 1 q ~ ; X 2 n \i>jX q ' n X 2 q ~ ) = because 
XL, is i.i.d. and is independent of the common randomness ui. Suppose that the statement of the lemma holds 
for i = j — 1, For i = j we proceed as follows: 

• If j is odd, we have 

/(x^-^xriwq^xrx^'-^^/c^x^^Sxriwq^.ijxrx^ 9 - 1 ) 

= /(X^-^Xflwqy.jjXfX^- 1 ) = 0, (86) 

where in the last step we use the first Markov chain of (f79) and the induction assumption. 

• If j is even, we have 

I(X 1 1:g -^X| :n |a;q 1:j] Xf n X 2 1:9 - 1 )<l(X 1 1:9 - 1 ;C' J X| :Tl |a;q 1:j _i ] Xf n X 2 1: «- 1 ) 

= /(X^- 1 ; Xn«qi:i-i]*f "X 2 1: «- x ) = 0, (87) 

where in the last step we use the second Markov chain of ((79} and the induction assumption. 
This completes the induction proof. ■ 
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